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Abstract 

A prime role of institutional 
research is to contextualize 
institutional or survey data 
in a clear, accurate, concise, 
and compelling fashion. The 
purpose of this paper is to 
delineate the strategy used 
by a comprehensive Canadian 
community college to improve 
the analysis and reporting of 
student satisfaction surveys. 
Although data-rich with many 
years of student satisfaction 
surveys, the community 
college was knowledge-poor 
as the data were only readily 
available in unsummarized 
form. Using exploratory factor 
analysis, the 44 questions on 
the surveys were reduced 
to eight dimensions. The 
data reduction technique 
facilitated the testing of 
relationships of student 
satisfaction with various 
institutional characteristics 
and student characteristics. 



The new variables were 
also used to prepare a new, 
public institutional research 
report, illustrating in graphic 
form satisfaction findings 
by program, department, 
and the College as a whole. 
The outcome was the 
greater use of the survey 
results by members of the 
College community, as well 
as prospective students and 
external agencies. 

Improving the 
Reporting of Student 
Satisfaction Surveys 
Through Factor 
Analysis 

Quality in education is 
a challenging concept to 
define, let alone to measure. 
Vroeijenstijn (1992) suggests 
that everyone in higher 
education has an interest or 
concern about quality, but 
not everyone has the same 
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idea about what it means. 
Bennett (2001) says quality is 
fundamentally what is improved 
about students' capabilities or 
knowledge as a result of their 
education. Tam (1999) argues 
that the assessment of quality 
in higher education should be 
measured in terms of impact on 
students' growth. Tan and Kek 
(2004) maintain that educational 
quality is determined by the 
extent to which students' needs 
and expectations are met. 
Indeed, quality in education 
may be characterized, in part, 
by the ability of an institution 
to satisfy students' needs and 
expectations (Tan & Kek, 2004). 

It is clear that higher 
education institutions are 
placing greater emphasis on 
student expectations and 
becoming increasingly aware 
of the importance of student 
satisfaction (Cheng &Tam, 1997; 
Elliot & Shin, 2002). Harvey 
(2001) argues student feedback 
plays an important role in 
assessing quality and standards. 
Astin (1993) suggests that 
student satisfaction is perhaps 
the key education outcome. 

This is supported by the finding 
that student satisfaction 
seems to have a positive 
impact on student motivation, 
student retention, and student 
recruitment (Elliot & Shin, 2002; 
Thomas & Galambos, 2004; 

Tinto, 1993). 

In addition, student 
satisfaction measures are widely 
used in advanced education 
with most postsecondary 
institutions undertaking 



such research, often through 
institutional research (Beltyukova 
& Fox, 2002; Downey, 2003). 
Nonetheless, there is criticism of 
the utility and validity of student 
satisfaction measures (Downey, 
2003), especially with instructor 
ratings (Greenwald, 1997). 
Admittedly, student satisfaction 
is a complex concept impacted 
by multiple characteristics 
of students and institutions 
(Thomas & Galambos, 2004). 
However, it seems clear 
that student satisfaction is 
a common measure used to 
assess educational quality and 
is beneficial to institutions 
in understanding student 
experiences and in assessing 
overall quality. 

For many years, our 
community college (Red River 
College or the College) in 
Western Canada has conducted 
an annual survey, the Student 
Evaluation of Program Survey 
(SEPS), of all students to 
measure satisfaction. Each 
year, students in their final 
term are surveyed to gather 
information on many aspects of 
the College. The questionnaire 
consists of a core of 1 1 
demographic questions, 44 
Likert-type attitude questions 
for all students, and additional 
questions for unique program 
features, such as co-operative 
work placement and clinical 
placement. A very large quantity 
of data is collected through the 
survey and reported for over 100 
programs. Traditionally, findings 
were reported in tables for each 
question for each program by 



year. Aggregate tables were also 
prepared by faculty and for the 
College as a whole. The data 
were distributed in large three- 
ring binders and available in MS 
Access files to senior academic 
leaders. However, given the bulk 
of the binders and the level of 
detail reported, full information 
was practically inaccessible to 
many constituencies such as 
students and faculty members. 
For example, the College- 
wide summary results alone 
were 12 pages in length, and 
the overall detailed tables 
ran for hundreds of pages. 
Although selected single- 
question items were used for 
trend analysis, full information 
was not used for planning, 
evaluation, and monitoring, or 
for accountability to external 
funders. As the fundamental 
role of institutional research in 
post-secondary education is to 
provide the analytic inputs to 
facilitate decision-making (Frost, 
1993; McLaughlin, Howard, & 
McLaughlin, 1998), a strategy 
was initiated to make the 
findings more accessible. 

Methods 

The data set for this research 
is the College's student 
evaluation of program survey 
results for 2003-04. A total of 
2,561 students from a potential 
population of 3,442 completed 
the survey questionnaire and 
are included in the data set. Like 
most surveys, this instrument 
includes many questions about 
one or more topics. Typically 
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how respondents answer these 
different questions tends to 
form patterns, that is, many of 
the responses are correlated. 

In order to understand the 
correlations and to summarize 
the findings in the 2003-04 
SEPS, the data were first reduced 
through factor analysis. This 
statistical technique allows the 
information contained in a large 
number of questions to be 
summarized in a smaller set of 
higher-order variables or factors. 
The main applications of factor 
analytic techniques are (a) to 
reduce the number of variables 
and (b) to detect and confirm 
structure in the relationships 
between variables, that is, to 
classify variables into categories 
which are relatively independent 
of each other. Factor analysis 
is a statistical approach used 
to analyze interrelationships 
among a large number of 
variables and to explain these 
variables in terms of their 
common underlying dimensions 
(Coughlin, 2005; Fisher & van 
Belle, 1993; Green & Salkind, 
2003; Pedhazur & Schmelkin, 
1991). The analysis compresses 
the original variables into a 
smaller set of underlying or 
basic factors. These factors 
consist of subcategories 
of the original variables 
correlated with one another 
but relatively independent of 
other subcategories of variables 
combined into other factors 
(Tabachnik & Fidell, 2001). 

There are two main types 
of factor analysis, exploratory 
and confirmatory. Exploratory 



factor analysis is generally used 
to discover the structure of a 
set of variables by grouping 
variables that are correlated. It is 
used when researchers have no 
hypotheses about the nature of 
the underlying factor structure 
of their measures. At this stage 
of research, researchers are 
primarily interested in data 
reduction, which is a common 
issue for institutional research. 

Confirmatory factor analysis is 
a more complicated statistical 
technique, perhaps best 
performed through structural 
equation modeling; it typically 
involves specialized statistical 
software, like AMOS. It requires 
the researcher to specify a factor 
structure in advance and to 
identify the variables to load 
on each factor (Coughlin, 2005; 
Pedhazur & Schmelkin, 1991). At 
this level of research, hypotheses 
about the factor structure are 
being tested. Confirmatory 
factor analysis is employed to 
test the validity of an identified 
factor structure. 

For this study, as there was 
no predetermined factor 
structure for the 2003-04 SEPS 
results, and because the prime 
interest was in data reduction, 
exploratory factor analysis 
was conducted. All of the 
attitude-type questions are on 
a four-point agree - disagree 
response scale. It may be argued 
that a four-point scale is not 
continuous; strictly speaking, 
it is ordinal. In practice, ordinal 
variables are often used in such 
analyses (Allison, 1999), and 
the basic requirement for factor 



analysis is to have variables 
at the ordinal level (Coughlin, 
2005). Such measures are 
sometimes referred to as "quasi- 
interval" (see Henkel, 1975; 
Labovitz, 1975). Moreover, factor 
analysis is very robust, and it is 
not uncommon to use factor 
analysis with four-point scales. 

Prior to undertaking 
exploratory factor analysis, the 
data set should be assessed to 
ascertain if it is appropriate for 
factor analysis. The factorability 
of a data set is evaluated 
through the Kaiser-Meyer- 
Olkin Measure (KMO) and 
Bartlett's Test of Sphericity. The 
KMO measures the sampling 
adequacy, which should be 
greater than 0.6 for factor 
analysis to be used (Tabachnik & 
Fidell, 2001). Sampling adequacy 
here measures whether the data 
are likely to factor well. If the 
data set is factorable, distinct 
factors will emerge. The KMO 
is computed as the ratio of the 
sum of squared correlations to 
the sum of squared correlations 
plus the sum of squared partial 
correlations. Values of the KMO 
range from 0 to 1 .0. The value 
approaches 1.0 if the partial 
correlations are small, meaning 
most variables measure a 
common factor (Tabachnik 
& Fidell, 2001). The Bartlett's 
Test of Sphericity examines 
whether there are adequate 
intercorrelations between the 
items to use factor analysis, and 
it should be significant (p < .05). 
This rejects the hypothesis that 
the correlations in a correlation 
matrix are zero (Tabachnik & 
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Fidell, 2001). These tests are 
readily available in SPSS and 
other analytic software. SPSS 
1 3.0 was used to conduct the 
analysis for this study. 

In brief, and with specific 
details of the study later, the 
way factor analysis works is 
that it traditionally starts from 
a correlation matrix for all the 
variables from the original 
data set. A correlation matrix 
is the rectangular array of 
the correlation coefficients of 
the variables with each other. 
There are three key steps in 
exploratory factor analysis: 

(a) extract the factors, (b) 
select the number of relevant 
factors, and (c) rotate the 
factors to maximize some set of 
relationships. Each of these will 
be elaborated in turn. 

The first step is to extract 
factors. Factors (dimensions) are 
extracted from the correlation 
matrix based on the correlation 
coefficients of the variables. 
Most factor analyses use one 
of the two main extraction 
methods, either Principal 
Components Analysis (PCA) 
or Common Factor Analysis 
(there are also several other 



specific techniques). SPSS and 
other analytic software provide 
these options for researchers. 
Both methods were used in 
conducting the exploratory 
factor analysis for this research. 
However, for purposes of 
reporting, the Common Factor 
Analysis is detailed in the body 
of this article while PCA is 
briefly outlined in an endnote.' 
The Common Factor Analysis 
method used is Principal Axis 
Factoring (PAF), which is argued 
to be the preferred method 
in recent literature (Coughlin, 
2005; Gorsuch, 1990; Preacher 
& MacCallum, 2003). In PAF, 
the primary or main diagonal 
(running from the top left corner 
to the bottom right corner) of 
the matrix contains estimates 
of the commonalities (the 
proportion of item variance 
that is in common with the 
other items). It uses squared 
multiple correlations for each 
item as the initial estimates of 
the commonalities (Pedhazur & 
Schmelkin, 1991). PAF analyzes 
common factor variability, 
that is, it attempts to account 
for common variance only. 

SPSS will provide a table of 



the commonalities. Initially 
the extraction process in PAF 
produces an equal number of 
factors with the initial variables 
and calculates an eigenvalue. 

An eigenvalue is the variability 
of a factor or the standardized 
variance associated with a 
particular factor (eigenvector). 
Eigenvalues in PAF are extracted 
sequentially by the amount of 
variance they explain, that is, the 
first factor will have the highest 
eigenvalue and so on to the last 
factor explaining the least. SPSS 
and other statistical software will 
produce output illustrating the 
total variance explained. At this 
point in a PAF, a researcher will 
have an array of eigenvalues. 

The second step is to select 
the number of relevant factors. 

In general, the number of 
dimensions or factors is much 
smaller than the number of 
original variables. Sometimes 
factors or dimensions or 
constructs are referred to as 
latent variables, particularly 
in psychological research, to 
distinguish them from what 
are referred to as measured, 
observed, manifest, or indicator 
variables. The latent concepts 



^Principal Components Analysis (PCA) with a Varimax rotation was also computed for these data. PCA is also a mathematical procedure to 
transform a large number of possibly correlated variables into a reduced number of uncorrelated variables which are referred to as principal 
components by some to distinguish the procedure from PAF. This is the default procedure in SPSS and is commonly employed (see Green & 
Salkind, 2003). PCA starts with ones on the principal diagonal. The 1.0 represents the relationship of the item with itself. These replace the 
commonalities used in PAF, and this means that the extracted factors account for all of the variability of each item, both common factor 
variability and the variance in the item that is idiosyncratic to the item. The steps in PCA are the same as for PAF. Using SPSS, PCA produces 
output with the eigenvalues of the components and a scree plot. Instead of a Pattern Matrix (in PAF) PCA produces a Component Matrix 
to use in assigning items to components. The rotation method used in this study was Varimax, an orthogonal technique. Varimax (variance 
maximizing) rotation is a method for rotating axes of a plot such that the eigenvectors remain orthogonal (that is uncorrelated) as they are 
rotated. This results in a situation where the sum of the variances of the loadings on a factor is the maximum possible. This rotation helps 
to simplify the interpretation of the components. Reviewing the Component Matrix resulted in similar components to the factors achieved 
through PAF. 
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or constructs are inferred from 
the directly measured variables, 
in this case, the 44 questions 
on the survey. Factors may 
be considered as manifesting 
the processes that form the 
correlations among the original 
variables (Tabachnik & Fidell, 
2001). As noted previously, 

PAF results in an array of 
eigenvalues from which to select 
the relevant number of factors. 
There are several methods to 
select factors (see Coughlin, 
2005; Green & Salkind, 2003; 
Tabachnik & Fidell, 2001). One 
is to use the Cattell Scree Test 
(plot) which may be produced in 
SPSS. A scree plot takes its name 
from the scree (rubble) at the 
bottom of a cliff and is a plot 
of the eigenvalues against their 
sequence number. It displays 
the highest eigenvalue, which 
is for the first factor, and then 
decreasing eigenvalues for 
the next factors until factors 
have small values. Figure 1 
shows the scree plot for our 
data. A criterion for selection 
of factors is to retain all factors 
with eigenvalues in the sharp 
descent part of the plot before 
the values level off. A second 
strategy, developed primarily for 
PCA is called the Kaiser-Guttman 
rule; it includes all factors with 
an eigenvalue greater than one. 
Another strategy to assist in 
selecting the number of factors 
is called Thurstone's Simple 
Structure (Tabachnik & Fidell, 
2001). If a simple structure is 
present, each factor has an 
adequate number of highly 
correlated variables (usually at 



least 3), and only one factor 
correlates highly with each 
variable. Thurstone's Simple 
Structure is used after factors 
are rotated. The scree test, the 
eigenvalue-greater-than-one 
criteria, and the Simple Structure 
Test are meant to act as guides 
in determining factors; it is 
also important to have a set of 
factors that can be identified 
from the data and that are 
meaningful (Fabrigar, Wegener, 
MacCallum, & Strahan, 1999). 
Researchers should always 
consider this before finalizing 
the number of relevant factors. 

The third step is to rotate the 
selected factors to maximize 
the relationship between the 
variables and the factors and 
to help in the interpretation 
of the factors. There are two 
main categories of rotation: 
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orthogonal and oblique. Briefly, 
orthogonal rotation assumes 
that there is no association 
between the factors. Orthogonal 
means at right angles, and when 
two variables are orthogonal, 
the correlation between them 
is zero (Pedhazur & Schmelkin, 
1991). Using an orthogonal 
rotation method in factor 
analysis retains this restriction. 
Oblique rotation allows the 
factors to be correlated (see 
Dillon & Goldstein, 1984). This 
method takes into account 
the relationships that may be 
present between factors. In 
this study, once the relevant 
number of factors was decided, 
the eigenvalues were rotated 
with a Promax rotation. Promax 
is an oblique rotation that allows 
factors to be correlated. With 
the SEPS data set, it is likely that 




Factor Number 



Figure 1. Scree plot for the principal axis factoring with eight factors. 
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the factors are correlated, as 
may be the case frequently with 
the type of surveys undertaken 
in institutional research. Some 
researchers (Fabrigar, et al., 

1999; Gorsuch, 1990; Preacher & 
MacCallum, 2003) suggest that 
PAF with a Promax rotation is 
the preferred method of factor 
analysis even if PCA is widely 
used and explained in many 
texts (for example. Green & 
Salkind, 2003). 

At this point, PAF will have 
produced several matrices. A 
factor correlation matrix displays 
the correlations among the 
factors; the structure matrix 
displays the correlations 
between factors and items; 
and the pattern matrix displays 
the unique contribution each 
factor makes to the variance 
of the item. The pattern matrix 
is the matrix traditionally used 
by researchers to determine 
which items fit with each 
factor. A general guideline is 
that factor loadings greater 
than .40 are considered to be 
useful (Coughlin, 2005). This is 
just a guideline and may need 
to be adjusted. For example, 
as the sample size and the 
number of variables increase, 
the guideline may need to be 
altered downward. It may need 
to be adjusted higher as the 
number of factors increases 
(see Hair, Anderson, Tatham, 

& Black, 1998). With SPSS, it 
is possible for researchers to 
specify that the pattern matrix 
only lists factor loadings over a 
certain value (for example .40) to 
facilitate interpretation. 



The researcher now will be 
able to identify which original 
items load (or fit best) on 
which factor. The nature of the 
original items will help interpret 
the dimension expressed by 
the computed factors and in 
the naming of the factors. The 
pattern matrix, as produced 
by SPSS, will only have factor 
numbers, and knowledge of 
the subject matter is needed 
to assign names to the factors. 
As previously mentioned, 
factor names should reflect the 
common element or underlying 
construct shared by the original 
items loading highly on the 
factor. 

After these steps, it is possible 
to compute factor scores for the 
selected factors. For each survey 
respondent, factors scores 
represent the estimates of the 
scores each respondent would 
have on each of the factors if 
the factors had been measured 
directly (Tabachnik & Fidell, 
2001). This allows the factors 
that represent the underlying 
or latent dimensions to be 
used in subsequent analyses. 
There are several procedures 
for estimating factor scores. A 
simple strategy is to sum and 
average scores on variables 
that load highly on each factor. 
This is useful for some relatively 
straightforward research 
purposes (Tabachnik & Fidell, 
2001) and may be used when 
the response scales for all items 
are the same. Other methods 
include the regression approach 
and the Anderson Rubin 
method, which are available 



in SPSS and other statistical 
software. These methods 
produce standardized scores 
that may be challenging to 
interpret and use in reporting of 
results, but they deal with issues 
of non-consistent scales and are 
useful if there is a lack of simple 
structure with items loading on 
more than one factor. 

Finally, the reliability of factors 
that each summarizes a number 
of items can be evaluated. In 
this study, once the factors 
were identified, new variables 
were created. The reliability of 
these new dimension variables 
was tested through Cronbach's 
alpha. Cronbach's alpha is not a 
statistical test; it is a coefficient 
of reliability (or consistency) 
that measures how well a set 
of items (or variables) measures 
a single unidimensional latent 
construct. The acceptable range 
is between .7 and 1.0 (Nunnally, 
1978). 

Results 

Table 1 shows that the data set 
from the 2003-04 SEPS satisfied 
the KMO Test at .946, confirming 
sampling adequacy. As noted, 
it should be greater than 0.6 
(Tabachnik & Fidell, 2001 ). The 
Bartlett's Test Chi square value 
of 1 1699.727 was significant; 
hence, the correlation matrix to 
be analyzed was non-random 
and was suitable for factor 
analysis. 

Examining the correlations 
among the many survey items in 
the 2003-04 SEPS revealed that 
there was a systematic pattern 
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Table 1 

KMO and Bartlett's Test Results for 2003-04 SEPS Factor 
Analysis 



Kaiser-Meyer-Olkin Measure of Sampling Adequacy 


.946 


Bartlett's Test of Sphericity 


Approx. Chi-Square 


11699.727 




df 


946 




Sig. 


.000 



of correlations within various 
subgroups of items. Initially 
seven factors were extracted 
with PAR The extraction was 
re-run for eight factors, as 
the original set of questions 
suggested an additional factor. 
SPSS and other analytic software 
allow researchers to select 
the number of factors to be 
extracted. As noted already, the 
scree test and the eigenvalue- 
greater-than-one criteria are 
meant to act as guides in 
determining factors. It is as 
important for researchers to use 
knowledge of the data in their 
assessment of the appropriate 
number of factors. In the case 
of this research, it was apparent 
that a small set of the questions 
in the survey spoke directly 
to orientation and, as this is a 
critical activity, the extraction 
was re-run to determine if these 
would separate out — and they 
did. In addition, in reviewing the 
full pattern matrix with eight 
factors, it was clear that the 
columns (the factors) had some 
high correlations and many low 
correlations, the columns had 
different patterns, and the rows 
(the items) had only one high 



value (see Tabachnik & Fidell, 
2001). This is the manner in 
which a factor pattern loading 
matrix is reviewed to assign 
items to factors. 

Figure 1 provides the scree 
plot for the PAF with eight 
factors. Table 2 provides part 
of the SPSS output for the 
eigenvalues (the first 15) with 
PAF. The first column under 
Initial Eigenvalues is the factor 
number, the second column is 
the eigenvalue for each factor; 
the third column is the percent 
of variance explained, and 
the final column is cumulative 
total of variance explained. The 
columns under Extraction Sums 
provide the values of the factor 
sum of the squared loadings. 
These values are somewhat 
different and smaller than the 
initial values as PAF was used, 
and these are a function of the 
item commonalities. 

Table 3 provides a summary 
of the items by factor (named 
based on an understanding 
of the original items) along 
with the loading for each item 
with its factor. The loadings 
came from the pattern matrix 
of the PAF factoring produced 



through SPSS. Our analysis 
achieved simple structure; 
each set of items, as illustrated, 
was a separate factor. Table 4 
provides the factor correlations 
(the correlations of factors with 
each other), which is a standard 
requirement to exhibit when 
using an oblique rotation. 

The eight factors were labeled 
satisfaction with the Program 
Overall (Program Quality); 
Orientation; Familiarization 
to College Policies (Policy 
Awareness); Welcoming, 

Inclusive College Environment; 
Academic Instruction; Program 
Resources; College Facilities; and 
College Services. For example. 
Academic Instruction was the 
name we gave one of the 
factors. Academic instruction is, 
of course, the core mission of 
educational institutions; hence, 
the Instruction factor will be 
examined to illustrate the logical 
coherence of the items and that 
the grouping of the items makes 
sense. There were four items in 
this factor, and they are grouped 
under the factor in Table 3. Each 
of these items speaks directly 
to instruction, and none of the 
other items related as directly. 

With the factor analysis 
complete and the factors 
named, factor scores were 
computed. As noted previously, 
factor scores can be computed 
through SPSS and other 
softwares; these factor scores 
compute individual scores for 
each respondent on each factor. 
SPSS will also save these scores 
as new variables for subsequent 
analysis. These factor scores. 
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Table 2 

Initial and Extracted Eigenvalues for the 2003-04 SEPS Data using Principal Axis Factoring 



Factor 


Initial Eigen 


values 




Extraction S 


urns of Squared Loadi 


ings 




Total 


% of Variance 


Cumulative % 


Total 


% of Variance 


Cumulative % 


1 


15.99 


36.341 


36.341 


15.591 


35.435 


35.435 


2 


3.100 


7.044 


43.386 


2.698 


6.131 


41.565 


3 


2.438 


5.542 


48.927 


2.098 


4.767 


46.333 


4 


2.309 


5.248 


54.175 


1.839 


4.180 


50.512 


5 


1.539 


3.498 


57.673 


1.140 


2.592 


53.104 


6 


1.299 


2.953 


60.626 


.834 


1.896 


55.000 


7 


1.188 


2.700 


63.326 


.798 


1.814 


56.815 


8 


.957 


2.176 


65.502 


.519 


1.179 


57.993 


9 


.885 


2.011 


67.512 








10 


.878 


1.995 


69.508 








11 


.802 


1.823 


71.331 








12 


.766 


1.741 


73.071 








13 


.670 


1.522 


74.594 








14 


.639 


1.452 


76.046 








15 


.592 


1.346 


77.392 









Note. Table truncated at factor 1 5. 



Table 3 

Summary of Items and Factor Loadings 



Loading 


Factor Name and Items 




Program Quality 


.551 


Before 1 applied, 1 had a good understanding of the program's purpose. 


.574 


The training 1 have received in this program has met my expectations. 


.720 


The program content is relevant to my career goals. 


.473 


The tuition fee for this program is reasonable for the education provided. 


.672 


Overall, 1 am satisfied with this program. 


.590 


1 would recommend this program to others. 




Orientation 


.433 


The orientation to the program provided by the Department was effective in explaining the requirements 
of the program. 


.447 


Upon admission to the program, 1 was made aware of my role and responsibilities as a student. 
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Table 3 (continued) 





Policy Awareness 


.756 


1 am familiar with the College's challenge for credit policy. 


.904 


1 am familiar with the College's transfer of credit policy. 


.922 


1 am familiar with the College's appeals procedure as it relates to academic and/or discipline issues. 


.722 


1 am familiar with the College's harassment policy. 


JA7 


1 am familiar with Prior Learning Assessment at the College. 




College Environment 


.768 


My gender does not limit my success in the program. 


J67 


My race or ethnic origin does not limit my success in the program. 


.830 


My physical ability does not limit my success in the program. 


.425 


My financial situation does not limit my success in the program. 


.612 


My English language skills do not limit my success in the program. 


.482 


My mathematical skills do not limit my success in the program. 


.418 


My experience in the program has increased my awareness of values and cultures that are different from my own. 




Instruction 


.779 


The instructors treat students with respect. 


.844 


The instructors are effective in delivering the program. 


.682 


The instructors are knowledgeable in the areas they teach. 


.828 


Overall, 1 am satisfied with the quality of instruction within the program. 




Program Resources 


.677 


The training materials (texts, workbooks, handouts, etc.) used in the program are current. 


.652 


1 am satisfied with the quality of the training materials used in this program. 


.788 


The equipment used in this program is appropriate for learning the required skills. 


.821 


The equipment used in this program is current with industry. 


.760 


There is a sufficient quantity of equipment provided for the program. 


.527 


There is a sufficient quantity of CURRENT library resource materials for use by students in the program. 




College Facilities 


.658 


The classroom facilities are appropriate. 


.635 


The shop/lab facilities are appropriate. 


.785 


Adequate study space is available to students. 


.723 


Student lounge space is adequate. 


.613 


The gymnasium/fitness facilities are satisfactory. 


.507 


Overall, the College facilities meet my needs as a student. 




College Services 


.571 


1 am satisfied with the service provided from the Academic Support Services (Tutorial Centre). 


.687 


1 am satisfied with the service 1 received from the Counselling Centre. 


.847 


1 am satisfied with the service 1 received from the Job Centre. 


.729 


1 am satisfied with the service 1 received from the Library. 


.754 


1 am satisfied with the service 1 received from the Bookstore. 


.860 


1 am satisfied with the service 1 received from the Enrolment Services Department. 


.818 


1 am satisfied with the service 1 received from the Print and Graphic Centre/Copy Centre. 


.768 


Overall, 1 am satisfied with the quality of service provided by the College. 
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Table 4 

Summary of Factor Correlations for the 2003-04 SEPS 



Factor Correlations 





College 

Services 


Policy 

Awareness 


Program 

Resources 


College 

Environment 


Instruction 


Program 

Quality 


College 

Facilities 


Orientation 


College Services 


1.000 
















Policy Awareness 


.549 


1.000 














Program Resources 


.659 


.500 


1.000 












College Environment 


.502 


.407 


.467 


1.000 










Instruction 


.416 


.410 


.518 


.402 


1.000 








Program Quality 


.458 


.552 


.561 


.382 


.623 


1.000 






College Facilities 


.706 


.486 


.692 


.443 


.469 


.494 


1.000 




Orientation 


.212 


.277 


.232 


.263 


.221 


.236 


.235 


1.000 



calculated, for example, through 
the regression method, are the 
most precise and capture the 
statistical relationships in the 
data. There are some concerns 
with using these scores. For 
example, they are not directly 
comparable with the original 
scale of the questions on the 
survey instrument, as they 
are standardized measures, 
computed from the factor 



score coefficient matrix. For the 
purposes of this study, as the 
original items all had the same 
response categories and all of 
the items loaded on a factor, 
the factors were calculated 
as summated mean scales to 
preserve comparability and for 
reporting (see Coughlin, 2005; 
Pedhazur & Schmelkin, 1991). 

As illustrated in Table 5, the 
dimensions or scale items 



were confirmed for reliability 
(.72 or higher) with Cronbach's 
alpha. One of the dimensions. 
Orientation, had only two 
variables. Simple structure 
would suggest, and Velicer and 
Fava (1998) argue, that factors 
should have at least three 
variables; however, if the original 
variables are best interpreted 
as a pair, and the intent is 
to develop the underlying 
dimensions, it makes sense to 
use only two. In addition, the 
two items were highly correlated 
(r = .6) and did not correlate 
nearly as highly with any other 
item (see Tabachnik & Fidell, 
2001 ). 

The new dimensions were 
used in a new Institutional 
Research report, illustrating 
in graphic form satisfaction 
findings by program, 
department, and the College 
as a whole (Red River College, 
2005). The first year report was 
about 55 pages in length, much 
shorter than the many hundreds 
of pages in the original set of 



Table 5 

Reliability of the Factors (Dimensions) Extracted from the 
2003-04 SEPS 



Dimension 


Reliability 


Number of Items 


Overall Program Quality 


.814 


6 


Quality of Orientation 


.721 


2 


Quality of Familiarization to 
College Policies 


.892 


5 


Quality of the Welcoming, Inclusive 
College Environment 


.794 


7 


Quality of Instruction 


.875 


4 


Quality of Program Resources 


.856 


6 


Quality of College Facilities 


.869 


6 


Quality of College Services 


.902 


7 
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Summary of Student Ratings of the College 

Program Quality 
Orientation 
Policy Awareness 
College Environment 
Instruction 
Program Resources 
College Facilities 
College Services 




0 0.5 1 1.5 2 2.5 

Mean Scale 



3.5 



Figure 2. Illustrative graph from the 2003-04 Student Evaluation of 
Program Report providing the Summary of Student Ratings for the 
College. 



three-ring binders. Figure 2 
provides an example of the 
College-wide results, which 
in traditional form was in 12 
pages of tables and now takes 
about one third of a page. 

The report itself is publicly 
available in electronic form and 
widely distributed in hard copy. 
The new variables have also 
provided the opportunity to 
conduct more readily detailed 
analyses, which facilitates 
understanding and acting on 
the influence of institutional 
and individual characteristics 
on satisfaction with different 
aspects of students' experiences. 

Discussion/Conclusion 

Using exploratory factor 
analysis improved the reporting 
of the student satisfaction 
survey. Forty-four original 
variables were summarized 



in eight factors reflecting 
key dimensions of student 
satisfaction with their college 
experience. Student satisfaction 
data are now widely accessible. 

A public report was prepared 
summarizing satisfaction 
using the dimension variables, 
illustrated in graphic form, at the 
college, faculty, and program 
levels. Survey data previously 
not available in summary form 
are now readily available and 
used by the President and 
other senior academic and 
administrative leaders of the 
College for decision-making. 

In addition, the College is able 
to demonstrate to students, 
through a public report, its 
commitment to listening to 
students' voices. 

The report has been well 
received and used by a variety 
of constituents, including the 



senior executive of the College, 
the Board of Governors, Senior 
Academic Committee, College 
Council, faculty, students, and 
external funding agencies. It 
is available at http://www.rrc. 
mb.ca. The program-level 
findings in the report are also 
directly linked to program 
descriptions on the College's 
web page, allowing prospective 
students to see the ratings of 
former students. The goal of the 
factor analysis exercise to find 
a credible, reliable summary of 
data that was acceptable to the 
whole college constituency was 
achieved. 

At a program level, it is 
now feasible to illustrate 
long-term trends in students' 
assessments in a form that 
includes all information but 
can be produced and reviewed 
in a manageable format. 
Furthermore, relationships 
among various institutional 
characteristics and student 
characteristics to different 
dimensions of satisfaction can 
now be tested with factors as 
the dependent variables. For 
example, differences in student 
satisfaction with the various 
dimensions by gender or 
academic type of program and 
delivery method can be readily 
examined through f-test or 
ANOVA. In addition, regression 
analysis with the overall 
program quality dimension as 
the dependent variable will help 
identify important predictor 
variables. 

Understanding student 
satisfaction is critical for 
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institutions of higher education. 
In a sense, student satisfaction 
is a key outcome by itself 
(Astin, 1 993; Okun & Weir, 

1990); however, it is also 
related to student performance 
(Pike, 1991) and to student 
persistence (Tinto 1993). Upcraft 
and Schuh (1996) suggest 
student satisfaction data help 
measure student outcomes 
in response to accountability 
demands and also signal that 
institutions are serious about 
responding to students' needs 
and improving effectiveness. 
Collecting information on 
satisfaction is necessary and 
so is mining that information 
for knowledge to help direct 
policies and action. Sanders 
and Chan (1996) outlined a 
methodology for conducting 
surveys and discussed various 
ways of maximizing the 
utility of the data. The use of 
factor analysis with student 
satisfaction data will enhance 
the role of institutional research 
in reporting findings and in 
identifying more completely 
the local, within-institution 
determinants of satisfaction 
to help identify needed 
improvements. 

The public report has 
become a staple of the work 
of the Institutional Research 
Department. A subsequent 
2004-05 Student Evaluation 
of Program Report has been 
evaluated using confirmatory 
factor analysis. As was indicated 
in the first part of this paper, 
factor analysis is usually 
conceived as having two stages: 



an exploratory stage and a 
confirmatory stage. The first 
stage, as discussed in this paper, 
identified a structure for the 
data. Some researchers have 
found it useful to apply further 
iterations of factor analysis to 
explore whether the structure 
can be simplified further. This 
approach is sometimes known 
as second-order factor analysis. 
Other researchers have moved 
directly to a confirmatory 
stage. Having identified a basic 
structure, it might be useful to 
explore whether a second-order 
model might add more insight 
to the results. Confirmatory 
factor analysis is a more 
statistically complex procedure 
used when a particular factor 
structure has been discovered 
or specified in advance, that 
is, the researcher designates 
the variables to load on each 
factor. In this situation, the 
factor structure discovered for 
the 2003-04 survey data was 
tested against the 2004-05 
survey data. Using AMOS (a 
structural equation modeling 
software distributed by SPSS), 
the 2003-04 factor structure was 
confirmed, which meant that 
the original eight factors worked 
with the 2004-05 survey data. 
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